Skip to main content

Matrix Decomposition

Matrix Decomposition (or Factorization) is simply breaking a big, complicated matrix down into smaller, simpler pieces.

The Cake Analogy​

Imagine someone hands you a baked cake. Matrix decomposition is like reverse-engineering the cake back into its raw ingredients: Flour, Eggs, and Sugar.

Why? Because raw ingredients are much easier to study, modify, and store than a fully baked cake! This is how algorithms like PCA and LoRA work!

Python Implementation (SVD)​

The most famous decomposition is Singular Value Decomposition (SVD). It breaks one matrix into three smaller ones (U,Σ,VTU, \Sigma, V^T).

import numpy as np

# Our complex "Cake" matrix
A = np.array([
    [3, 1, 1],
    [-1, 3, 1]
])

# Decompose the cake into ingredients (U, S, V)
U, S, VT = np.linalg.svd(A)

print("Ingredient U (Rotation):\n", U)
print("Ingredient S (Stretching):\n", S)
print("Ingredient V (Rotation):\n", VT)